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Abstract 



l_J ' Testing is a vital part of the software development process. Test Case Generation (TCG) 

Q, , is the process of automatically generating a collection of test-cases which are applied to a 

• ■ system under test. White-box TCG is usually performed by means of symbolic execution, 

fj ' i.e., instead of executing the program on normal values (e.g., numbers), the program is 

executed on symbolic values representing arbitrary values. When dealing with an object- 
oriented (OO) imperative language, symbolic execution becomes challenging as, among 
other things, it must be able to backtrack, complex heap-allocated data structures should 
be created during the TCG process and features like inheritance, virtual invocations and 
exceptions have to be taken into account. Due to its inherent symbolic execution mecha- 
nism, we pursue in this paper that Constraint Logic Programming (CLP) has a promising 
I/-S ■ application field in TCG. We will support our claim by developing a fully CLP-based 

^^" ' framework to TCG of an OO imperative language, and by assessing it on a corresponding 

I— , implementation on a set of challenging Java programs, 

j.-^ ' KEYWORDS: Test case generation. Symbolic execution, Constraint logic programming 



^^ ' 1 Introduction 

H , 

5t 1 Test Case Generation (TCG) is the process of automatically generating a collec- 

tion of test-eases which are applied to a system under test. The generated cases 
must ensure a certain coverage criterion (see e.g., (IZhu et al. 1997[) for a survey) 
which are heuristics that estimates how well the program is exercised by a test 
suite. Examples of coverage criteria are statement coverage, which requires that 
each line of the code is executed, path coverage which requires that every pos- 
sible trace through a given part of the code is executed, loop-k (resp. block-k) 
which limits to a threshold k the number of times we iterate on loops (resp. 
visit blocks in the control flow graph () Albert et al. 2009]) ). Among all possible 
forms of TCG, we focus on static (i.e., no knowledge about the input data is as- 
sumed) and white-box TCG (i.e., the program is used for guiding the TCG process). 
The standard way of performing static white-box TCG is program symbolic exe- 
cution (SymEx) ( |King 1976[ IGotlieb et al. 2000( IMeudec 20011 IMtiUer et al. 2004| 
ITillmann and de Halleux 2008p . whereby instead of on actual values, programs are 
executed on symbolic values, sometimes represented as constraint variables. Such 
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class SLNode { 


// loop 


int data; 


while ((pi != null) kk (p2 != null)){ 


SLNode next; 


// loopcondl, loopcond2 and loopbodyl 


} 


if (pi. data <= p2.data){// «/2 




curr. next = pi; 


class SortedList { 


pi = pi. next; 


SLNode first; 


} 




else { 


public void merge (SortedList 1){ 


curr. next = p2; 


SLNode pl,p2,curr; 


p2 = p2.next; 


pi = first; p2 = Q) 1. first; 


} 


if ((2) pi. data <= (3) p2.data)// «/i 


curr = cuTT .next; // loopbody 2 


pi = (3) pi. next; 


} 


else { 


if (pi == null) curr. next = p2;//i/3 


first = p2; p2 = p2.next; } 


else curr. next = pi; 


curr = first; // preloop 


} 



Fig. 1. Working example: Java source code 



constraints arc accumulated into path constraints as each path of the execution 
tree is expanded. The path constraints in feasible paths provide pre-conditions on 
the input data which guarantee that the corresponding path will be executed at 
run-time. 

In this paper, we pursue that Gonstraint Logic Programming (CLP) has a promis- 
ing application field in TCG, since it inherently combines the use of constraint 
solvers into its SymEx mechanism. Our main goal is to formalize a whole TCG 
framework for a realistic object-oriented (00) imperative language by means of 
CLP. Our approach consists of two basic parts: first, the imperative program is 
compiled into an equivalent CLP program and, second, TCG is performed on the 
CLP program by relying only on CLP's evaluation mechanisms. The main chal- 
lenges in TCG when dealing with an 00 imperative language are related to the 
creation of complex heap-allocated data structures during the TCG process, and 
to handling 00 features like inheritance and virtual invocations, and exceptions. 
Besides, when dealing with objects, one needs to take into account all possible 
aliasing among them, since this might affect directly the coverage of the test-cases. 
Previous approaches strive to define novel specific constraint operators to carry 
out these tasks (see e.g. (jCharreteur et al. 2009| [Schrijvers et al. 2009[ )). Instead, 
in our approach, the whole TCG process is formulated using CLP only, and with- 
out the need of defining specific operators to handle the different features. This, on 
one hand, has the advantage of providing a clean and uniform formalization. And, 
more importantly, since SymEx is performed on an equivalent CLP program, we 
can often obtain the desired degree of coverage by using existing evaluation strate- 
gies on the CLP side. This gives us flexibility and parametricity w.r.t. the adequacy 
criteria. 

Our approach has been integrated in PET (jAlbert et al. 20l"0|) . a Partial- Evaluation 
based TCG tool, extending its applicability towards real-life 00 applications. 
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2 A CLP-Executable Object-Oriented Imperative Language 

In this section, we define tlie (CLP) syntax and semantics of tlie 00 imperative 
language on which our TCG approach is developed, which we call CLP-decompiled 
language. Its main characteristic is that it keeps all features of the original 00 
language but it is CLP-executable, i.e., it can be executed using the evaluation 
mechanism of CLP languages. When the source imperative language is low-level 
as bytecode, we use the term CLP-decompiled language. In previous work, it has 
been shown that Java bytecode (and hence Java) can be decompiled into a simi- 
lar language (jGomez-Zamalloa et al. 2009[) by relying on the interpretive approach 
(jFutamura 197ip to compilation, proposed in the first Futamura projection. In this 
approach, the CLP-(de)compilation is obtained by partially evaluating an inter- 
preter for the 00 language written in CLP. 

Example 1 

Fig. [T] shows the source code of our running example which implements a merge 
algorithm on sorted singly-linked lists. Fig. [2] shows the CLP-decompiled program 
automatically generated by our system from the bytecode obtained by compiling the 
Java program, with some simplifications to improve readability. The correspondence 
between blocks of the original program and clauses in the decompiled one is shown 
in comments in the Java code. The main features that can be observed from the 
decompilation are: (1) All clauses contain input and output arguments and heaps, 
and an exceptional flag. As in the bytecode, input arguments of non-static methods 
include the reference this (named r(Th)). Reference variables are of the form r(V) 
and we use the same variable name V as in the program. (2) Java exceptions are 
made explicit in the decompiled program. Observe predicates nullcheckx, which 
capture the exceptions that can be thrown at program points annotated as ®. (3) 
Conditional statements in the source program arc transformed to guarded rules 
in the CLP one (e.g., ifl). (4) Iteration in the source program is transformed into 
recursion in the CLP program. E.g, the while loop corresponds to the recursive 
predicate loop. 



2.1 Syntax of CLP -Decompiled Object-Oriented Imperative Programs 

As illustrated in Fig. [21 a CLP-decompiled program consists of a set of predicates. 
A predicate p is defined by one or more clauses which are mutually exclusive. This 
is ensured, either by means of mutually exclusive guards, or by information made 
explicit on the clause heads (as usual in CLP). Each clause p receives as input a 
(possibly empty) list of arguments Argsin and an input heap Hin, and returns the 
(possibly empty) output Argsouti a possibly modified output heap Hout, and an 
exception flag. This flag indicates whether the execution ends normally or with an 
uncaught exception. Clauses adhere to the following grammar. As usual, terminals 
start by lowercase (or special symbols) and non-terminals by uppercase. Subscripts 
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merge ([[r(Th),L] , [] ,Hi„ ,Hout ,EF) :- getJield(Hin,Th, 'SL' :first,Pl), 
nullcheckl([r(Th) ,L,P1] , [] ,Hi„ ,Hout,EF) . 

nullcheckli([r(Th),r(L),Pl] , [] ,H1,H2,EF) :- getJield(Hl ,L, 'SL' :first,P2), 

nullcheck2([r(Th),r(L),Pl,P2] , [] ,H1,H2,EF) . 
nullcheckl2([r(_),null,_] , [] ,H1 ,H2,exc(ExRef ) ) :- new_object(Hl , 'NPE' ,ExRef ,H2) . 

nullcheck2i([r(Th),r(L),r(Pl),P2] , [] ,H1,H2,EF) :- getJield(Hl ,P1, 'SL' :data,Datal) , 

iiullcheck3([Datal,r(Th) ,r(L) ,r(Pl) ,P2] , [] ,H1,H2,EF) . 
nullcheck22([r(_),r(_),null,_] , [] ,H1 ,H2,exc(ExRef ) ) :- new_object(Hl, 'NPE' ,ExRef ,H2) . 

nullcheck3i([Dl,r(Th),r(L),r(Pl),r(P2)] , [] ,H1,H2,EF) :- get_field(Hl,P2, 'SL' :data,D2), 

ifl([D2,Dl,r(Th),r(L),r(Pl),r(P2)] , [] ,H1,H2,EF). 
nullcheck32([-,-,r(_),r(_),null] , [] ,Hl,H2,exc(ExR) ) :- new_object(Hl, 'NPE' ,ExR,H2) . 

lfll([Data2,Datal,r(Th),r(L),r(Pl),r(P2)] , [] ,H1,H3,EF) :- Datal #> Data2, 
setJield(Hl,Th, 'SL' :flrst,r(P2) ,H2) , getJield(H2,P2, 'SL' :next,P2') , 
preloopC [r(Th) ,r(L) ,r(Pl) ,P2'] , [] ,H2,H3,EF) . 

ifll([Data2, Datal, r(Th), r(L), r(Pl), r(P2)] ,[], HI, H2,EF) :- Datal #=< Data2, 

get_field(Hl,Pl,'SL':next,Pl'), preloopC [r(Th) ,r(L) ,P1' ,r(P2)] , [] ,H1,H2,EF). 

preloopC [r(Th) ,L,P1,P2] , [] ,H1,H2,EF) :- 

get_field(Hl,Th,'SL':first,Curr), loop( [r(Th) ,L,Pl,P2,Curr] , [] ,H1,H2,EF). 

loop([Th,L,Pl,P2,Curr] , [] ,H1,H2,EF) :- loopcondK [Th,L,Pl,P2,Curr] , [] ,H1,H2,EF). 
loopcondli ([_,_, null, P2,Curr] , [] ,H1,H2,EF) :- if 3( [null,P2,Curr] , [] ,H1,H2,EF). 
loopcondl2([Th,L,r(Pl),P2,Curr] , [] ,H1,H2,EF) :- 

loopcond2([Th,L,r(Pl),P2,Curr] , [] ,H1,H2,EF). 

loopcond2i([_,_,r(Pl),null,Cur] , [] ,H1 ,H2,EF) :- if 3 ( [r (PI) , null. Cur] , [] ,H1,H2,EF). 
loopcond22([Th,L,r(Pl) ,r(P2) ,Curr] , [] ,H1,H2,EF) :- 

loopbodyl([Th,L,r(Pl),r(P2),Curr] , [] ,H1,H2,EF) . 

loopbodyl([Th,L,r(Pl) ,r(P2) ,Curr] , [] ,H1,H2,EF) :- 

get_field(Hl,Pl, 'SL' :data, Datal) , get_field(Hl,P2, 'SL' :data,Data2) , 
if2([Data2,Datal,Th,L,r(Pl),r(P2),Curr] , [] ,H1,H2,EF). 

if2i([Data2, Datal, Th, L, r(Pl), r(P2), r(Curr)] ,[], HI, H3,EF) :- Datal #> Data2, 

set Jield (HI , Curr , ' SL ' : next , r (P2) , H2) , getjield (H2 , P2 , ' SL ' : next , P2 ' ) , 

loopbody2([Th,L,r(Pl),P2',r(Curr)] , [] ,H2,H3,EF). 
if22([Data2, Datal, Th, L, r(Pl), r(P2), r(Curr)] ,[], HI, H3,EF) :- Datal #=< Data2, 

setJield(Hl,Curr,'SL':next,r(Pl),H2), get_field(H2,Pl, 'SL' :next,Pl'), 

loopbody2([Th,L,Pl',r(P2),r(Curr)] , [] ,H2,H3,EF). 

loopbody2([Th,L,Pl,P2,r(Curr)] , [] ,H1,H2,EF) :- 

get_field(Hl,Curr, 'SL' :next,Curr') , loop( [Th,L,Pl,P2,Curr'] , [] ,H1,H2,EF) . 

if3i([r(Pl),_,r(Curr)] , [] ,H1 ,H2,ok) :- set_field(Hl,Curr , 'SL' :next ,r(Pl) ,H2) . 
if32( [null, P2,r (Curr)] , [] ,H1 ,H2,ok) :- set_field(Hl,Curr , 'SL' :next ,P2,H2) . 



Fig. 2. Working example: CLP-decompiled code 



are provided just for clarity. 



Clause 
G 
B 



= Pred {Argsin,ArgSout,Hi„,Hout,ExFlag) :- [G,]Bi,B2,. . . ,Bn. 

= Num* ROp Num.* \ Ref^ \== Ref2 \ ty pe{H, Ref ,T) 

= Vari^= Num* AOp Num* \ Pred {Argsi„,ArgSont,Hi„,Hout,ExFlag) \ 

new.ob}ect{H,C ,Ref ,H) \ n&NjSimy{H,T,Num' ,Ref ,H) \ \ength{H,Ref ,Var) 
getJ\e\d{H,Ref ,FSig,Var) \ setJ\e\d{H,Ref ,FStg,Data'' ,H) \ 
get^nay{H,Ref ,Num* ,Var) \ set^rray{H, Ref , Num* .Data* ,H) 
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Pred 


■- Block 1 MSig 


ROp: 


~ #>!#<! #>= 1 #=< 1 #= 


l#\= 


Args 


;=[] 1 [Data* \ Args] 


AOp: 


:=+-*!/! mod 




Data 


:= Num. 1 Ref 


T: 


:= bool \ int\ C \ array(T) 




Ref 


:= null 1 r(Var) 


FSig: 


■- O.FN 




ExFlag 


:= ok exc(Var) 


H: 


■- Var 





Non-terminals Block, Num, Var, FN, MSig and C denote, resp., the set of predicate 
names, numbers, variables, field names, method signatures and class names. Observe 
that clauses can define both methods which appear in the original source program 
(MSig), or additional predicates which correspond to intermediate blocks in the 
program (Block). An asterisk on a non-terminal denotes that it can be either as 
defined by the grammar or a (possibly constraint) variable. Guards might contain: 
comparisons between numeric data or references and calls to the type predicate, 
which checks the type of a reference variable (by consulting the heap). Virtual 
method invocations in the 00 language are resolved at compile-time by looking up 
all possible runtime instances of the method. In the decompiled program, they 
are translated into a choice of type instructions which check the actual object 
type, followed by the corresponding method invocation for each runtime instance. 
Instructions in the body of clauses include: (first row) arithmetic operations, calls 
to other predicates, (second row) instructions to create objects and arrays, and to 
consult the array length, (third row) read and write access to object fields, and, 
(fourth row) read and write access to an array position. As regards exceptions, they 
can be handled by treating them as additional nodes and arcs in the control flow 
graph of the program. In our framework, such flows are represented in the CLP- 
decompiled program with explicit calls to the corresponding exception handlers. 

For simplicity, the language does not include features of 00 imperative lan- 
guages like bitwise operations, static fields, access control (e.g., the use of public, 
protected and private modifiers) and primitive types besides integers and booleans. 
Most of these features can be easily handled in this framework, as shown by the 
implementation based on actual Java bytecode. 

2.2 Semantics of CLP- Decompiled Programs with Heap 

When considering a simple imperative language without heap-allocated data struc- 
tures, like in (I Albert et al. 2009)) . CLP-decompiled programs can be executed by 
using the standard execution mechanism of CLP. In order to extend this approach 
to a realistic language with dynamic memory, as our first contribution, we provide 
a suitable representation for the heap and define the heap related operations. Note 
that, in CLP-dccompilcd programs the heap is treated as a black-box through its 
associated operations, therefore it is always a variable. At run-time, the heap is 
represented as a list of locations which arc pairs made up of a unique reference 
and a cell, which in turn can be an object or an array. An object contains its type 
and its list of fields, each of them contains its signature and data contents. An 
array contains its type, its length and the list of its elements. Observe that arrays 
are stored in the heap together with objects (as it happens e.g. in Java bytecode). 
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build_object(C,Ob), new_ref(Ref), H' = [(Ref,Ob)|H]. 
build_array(T,L,Arr), new_ref(Ref), H' = [(Ref,Arr)lH]. 

get_cell(H,Ref,Cell), Cell = object(T,_). 
get_cell(H,Ref,Cell), Cell = array(_,L,_). 

get_cell(H,Ref,Ob), FSig = C:FN, Ob = object(T,Fields), 
subclass(T,C), member_det(field(FN,V), Fields). 
get_array(H,Ref,l,V) :- get_cell(H,Ref,Arr), Arr = array(_,_,Xs), nthO(l,Xs,V). 

set.field(H,Ref,FSig,V,H') :- get_cell(H,Ref,Ob), FSig = C:FN, Ob = object(T, Fields), 

subclass(T,C), replace_det(Fields,field(FN,_),field(FN,V),Fds'; 
set_cell(H,Ref,object(T,Fds'),H'). 
set_array(H,Ref,l,V,H') :- get_cell(H,Ref,Arr), Arr = array(T,L,Xs), 

replace_nthO(Xs,l,V,Xs'), set_cell(H,Ref,array(T,L,Xs'),H'). 

get_cell([(Ref',Cell')|_],Ref,Cell) :- Ref == Ref, !, Cell = Cell'. 
get_cell([_|RH],Ref,Ob) :- get_cell(RH,Ref,Ob). 

set_cell([(Ref',_)|H],Ref,Cell,H') :- Ref == Ref, !, H' = [(Ref,Cell)|H]. 
set_cell([(Ref' ,Cell')|H'],Ref,Cell,H) :- H = [(Ref ,Ceir)|H"], set_cell(H', Ref Cell, H"). 



new_object(H,C,Ref H') 
new_array(H,T,L,RefH') 

type(H,RefT) 
length(H,RefL) 

get_field(H,RefFSig,V) 



Fig. 3. Heap operations for ground execution 

Formally, the syntax of the heap at run-time is as follows. The asterisks will be 
explained later: 

Heap ::— [] | [Loc\Heap] Cell ::= object(C* , Fields*) \ array{T^ ,Num* ,Args*) 

Loc :■- {Num*,Cell) Fields ::= [] | [field{FN,Data)\Fields''\ 

In the upper side of the figure, we present the CLP-implementation of the oper- 
ations to create heap-allocated data structures (like new_object and new_array) and 
to read and modify them (like setJield, etc.), and, at the bottom appear some 
auxiliary predicates. To simplify the presentation some predicates are omitted, 
namely: build_object/2 resp. build_array/3, which create an object, resp. an array 
term, new_ref/l which produces a fresh numeric reference, and subclass/2 which im- 
plements the transitive and reflexive subclass relation on two classes. member_det/2 
resp. replace_det/4 implements the usual deterministic member, resp. replace, on 
lists, while nthO/3 resp. replace_nth0/3 implements the access to, resp. replacement 
of, the i*'' clement of a list using constraints (multi-moded versions). 

We now focus on the ground execution of CLP-decompiled programs in which 
we assume that all input parameters of the predicate to be executed (i.e., Argsin 
and Hin) are fully instantiated. The instantiations are provided as constraints in 
the input state. We assume familiarity with the basic notions of CLP. Very briefly, 
let us recall that the operational semantics of a CLP program P can be defined 
in terms of derivations, which arc sequences of reductions between states So -^p 
Si — >p ... -^p Sn, also denoted 5*0 ^-p S'„, where a state {G I 9) consists of a goal G 
and a constraint store 0. If the derivation successfully terminates, then S'„ = (e I 6') 
and 0' is called the output state. 

Definition 1 (ground execution) 

Let M be a method, m be the corresponding predicate from its associated CLP- 
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decompiled program P, and P' be the union of P and the clauses in Fig. [3l The 
ground execution of m with input 9 is the derivation Sq -^p' Sn, where So = 
{in{Argsin,ArgSout,Hin,Hout,ExFlag) I 9) and 9 initializes Argsm and Hin to be 
fully ground. If the derivation successfully terminates, then Sn = (e I 9') and 9' is 
the output state (e denotes the empty goal). 

Every CLP-decompilation must ensure that CLP programs capture the same se- 
mantics of the original imperative ones. This is to say that, given a correct in- 
put state, the CLP-execution yields an equivalent output state. By correct input 
state, we mean that all input arguments have the correct types and that the 
heap has the required contents. For instance, 9 = {ArgSi„ — [r(l),null] A Hi„ — 
[(l,object('SL',[field('SL':first,null)]))]} is a correct input state for predicate merge/5, 
whereas 9 — {ArgSm = [r(l),r(2)] A Hi„ — []} is not correct since the heap docs not 
include the required objects. 

Definition 2 {correct decompilation) 

Consider a method M and a correct input state /. Let m be the CLP-decompiled 
predicate obtained from M and 9 be the input state equivalent to /. If the CLP- 
decompilation is correct then it must hold that, the execution in the 00 language 
of M returns as output state O if and only if the ground execution of m with 9 is 
deterministic and returns an output state 9' equivalent to 0. 

Correctness must be proven for the particular techniques used to carry out the de- 
compilation. In the interpretive approach, for a simpler bytecode language without 
heap, (jGomez-Zamalloa et al. 2009| proves that the execution of the decompiled 
programs produces the same output state than the execution of the bytecode pro- 
gram in the CLP interpreter. A full proof would require to prove that the CLP 
interpreter is correct and complete w.r.t the corresponding imperative language se- 
mantics. Since our approach is not tied to a particular decompilation technique, in 
the rest of the paper, for the correctness of our TDG approach, we just require that 
decompiled programs are correct as stated in Def. [2] 

Finally, in the above definition, it can be observed that, since CLP-decompilcd 
programs originate from imperative bytecode, their ground execution is determinis- 
tic. The aim of the next section is to be able to execute CLP-decompiled programs 
symbolically with the input arguments being free variables. 



3 Symbolic Execution of OO Imperative Programs 

Interestingly, our CLP-decompilcd programs can in principle be used, not only 
to perform ground execution, but also symbolic execution (SymEx). Indeed, when 
the imperative language does not use dynamic memory nor 00 features, we can 
simply run the CLP-decompilcd programs by using the standard CLP execution 
mechanism with all arguments being distinct free variables. For simple imperative 
languages, this approach was first proposed by (jMeudec 200ip and developed for a 
simple bytecode language in (j Albert et al. 2009]) . However, dealing with dynamic 
memory and 00 features entails further complications, as we show in this section. 
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3.1 Handling Heap- Allocation in Symbolic Execution 

In principle, SyniEx starts with a fully unknown input state, including a fully un- 
known heap. Thus, one has to provide some method which builds a heap associated 
with a given path by using only the constraints induced by the visited code. In the 
case of TCG, it is required that the ground execution with that heap (and the corre- 
sponding input arguments) traverses exactly such path. Existing approaches define 
novel specific operators to carry out this task. For instance, (jCharreteur et al. 2009P 
adds new constraint models for the heap that extend the basic constraint-based ap- 
proach without heap. Similarly, ( [Schrijvers et al. 2009 ) provides specific constraints 



for heap-allocated lists, but needs to adjust the solver to handle other data struc- 
tures. In our approach, thanks to the explicit representation of the heap, we are 
able to provide a general solution for the SymEx of programs with arbitrary heap- 
allocated data structures. 

The main point is that in a ground execution, the heap is totally instantiated 
and, when we execute get_cell/3 (sec Fig. [3|), the reference we are searching for 
must be a number (not a variable) existing in the heap. In contrast, SymEx deals 
with partially unknown heaps. Our solution consists in generalizing the definition 
of get_cell/3 by adding an additional clause (the first one) as follows: 



get_cell(H,Ref,Cell) 

get_cell([(Ref',Cell')|_],Ref,Cell) 

get_cell([_jRH],Ref,Cell) 



var(H), !, H = [(Ref,Cell)|_ 
Ref == Ref, !, Cell = Cell 
get_cell(RH,Ref,Cell). 



Intuitively, the heap during SymEx contains two parts; the known part, with the 
cells that have been explicitly created during SymEx which appear at the beginning 
of the list, and the unknown part, which is a logic variable (tail of the list) in 
which new data can be added. Observe the syntax of the heap in Sect. 12.21 where 
the *'s indicate where partial information can occur in the heaps during SymEx. 
Such syntax is hence valid for all heaps appearing at SymEx time. The definition 
of get_cell/3 now distinguishes two situations when searching for a reference: (i) 
It finds it in the known part (second clause). Note the use of syntactic equality 
rather than unification since references at SymEx time can be variables or numbers, 
(ii) Otherwise, it reaches the unknown part of the heap (a logic variable), and it 
allocates the reference (in this case a variable) there (first clause). 

Example 2 

Let us use our SymEx framework for the purpose of TCG on our working exam- 
ple. As will be further explained, for this it is required to: (i) impose a termi- 
nation criterion on SymEx, and (ii) have a mechanism to produce actual values 
from the obtained path constraints. For (i) let us use block-fc with K = 2. Re- 
garding (ii), we just rely on the labeling mechanism of standard clpfd domains, 
since we only get arithmetic path constraints. The rest of the constraints are 
handled as explained with standard unification through the defined heap oper- 
ations. Table [T] depicts a graphical representation of the obtained set of test- 
cases. The table shows, for each test-case, an identifier, a graphical representa- 
tion of its input and output, and the exception flag. Due to space limitations, 
we do not show the full input and output heaps, but instead we use the custom- 
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N Input 



Output 



EF 



1 this.first^Hj) ^ C l.first^^cTH^ null this.first^»(o) 



ok 



■C ok 



3 this.first^(T) — s* null 

l.first-HA>— HT)-^- C 



■C ok 



4 this.first^»(o)-^- null l.first— *(o) — s- C this.first^^JD) 



ok 



5 this.first^V) HT) ^ C 

Lfirst-H^H" null 



7 this.first^Hj) ^ C l.first = null 



this. first— *- null l.first- 



this. firsts C I — s- null 



Table 1. Obtained test-cases for working example 

ary graphical representation for the linked lists of integers that they contain (see 
the example below to understand the correspondence). Let us focus on the first 
test-case. It corresponds to the following (simplified) sequence of reduction steps 
merge — S^nullchecki — >■ nullcheck2— !■ nullchecks^- if li— ^ preloop ^>loop — !>loopcondl2— ^ 
->loopcond2i^ ifSi. Its associated answer is 9 = {Argsi„ = [r(Th),r(L)] A Hi„ — [(Th, 

object('SL',[field(first,A)])), (L,object('SL',[field(first,B)])),(A,object('SLNode',[field(data,l)])), (B, ob- 

ject('SLNode',[field(data,0),field(next,null)]))] A . . .}, indicating that merging a list with head 
"1" and any possible continuation (denoted "C"), and a null-terminated list with 
head "0" , produces an output list with head "0" , followed by "1" and followed by 
the continuation "C'.The last three test-cases show that, either if 1 is null, or the 
first field of any of the lists is null, the method throws an exception. This is indeed 
spotting a bug in the program (assuming it is not the intended behavior). 



3.2 Handling Pointer Aliasing in Symbolic Execution 

A challenge in SymEx of realistic languages is to consider jtointer- aliasing during 
the generation of heap-allocated data structures, i.e., the fact that the same memory 
location can be accessed through several references (called aliases). In the case of 
TCG, ignoring aliasing can lead to a loss of coverage. Again, our solution consists 
in further generalizing the definition of get_cell/3 by adding an additional clause 
(the third one), thus illustrating again the fiexibility of our approach: 



get_cell(H,Ref,Cell) 
get_cell([(Ref',Cell')|_],Ref,Cell) 
get_cell([(Ref',Cell')|_],Ref,Cell) 
get_cell([_|RH],Ref,Cell) 



var(H), !, H = [(Ref,Cell)|_]. 
Ref == Ref, !, Cell = Cell'. 
var(Ref), var(Ref'), Ref = Ref 
get_cell(RH,Ref,Cell). 



Cell 



Cell'. 



Essentially, two cases are distinguished: (a) The reference we are searching for is a 
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Input Output EF 



10 this.first^O) — ^ null I = this this.first^^O)--^ ok 




11 this.first-HO) HOV^-null I = this this.first^OV^^-NO) ok 



I = this this.first^>(C^-^^^{0) 



12 this. firsts null I = this 



13 this.first-^O) — ^ null this.first^O>~^ ok 

I. first' 



14 this.firstj(o) ^-(o)—^ null this.first^^T) >(o) ok 



15 this.first-s-(l)^ null this.first^^O) Ki^ "^ 

l.first-^(0K XJ 



16 



this.first-H^oV this.first^^T) Ko)-v °^ 

\.fWst-^(0f^ null 



Table 2. Additional test-cases when considering pointer-aliasing 

number, in that case it must exist in the heap and the 2nd clause will eventually 
succeed, (b) If Ref is a variable: (b.l) Ref exists in the heap, and the 2nd clause 
eventually succeeds. Here, Ref must have been already processed (and possible 
aliases for it might have been created, (b.2) The interesting case is when Ref is 
a free variable which was not in the heap. In this case, the 2nd clause will never 
succeed and the 3rd one will unify Ref with all matching references in the heap. 

Example 3 

Let us consider again the TCG for our working example as in Ex. [2] Tabic [2] shows 
seven additional test-cases obtained using the new definition of get_cell/3. Test- 
cases 10-12 represent executions in which the two lists to be merged are aliases. The 
remaining test-cases show other shapes of lists with aliasing among their nodes. In 
most cases, the result is a cyclic list. This clearly reveals a dangerous behavior of 
the method which should be controlled by the programmer. Altogether, our set of 
test-cases provides full coverage w.r.t. the shape of data structures. 

3.3 Inheritance and Virtual Invocations in Symbolic Execution 

Inheritance and virtual method invocations pose further challenges in SymEx of 
realistic 00 programming languages. From the side of data structure shape cover- 
age, we should create aliasing among objects that possibly have different class types 
but, due to their inheritance relation, might be aliased at runtime. From the side of 
path coverage, virtual invocations pose further complications when the object on 
which the virtual invocation is performed has not been created during SymEx, but 
is rather accessed from the input arguments. In this case, only the declaration type 
of the object is known. To achieve path coverage, all implementations of the method 
that might be invoked at runtime (but not more), should be exercised. Interestingly, 
our solution solves these issues for free. Let us consider a scenario where we have 
three classes A, B and C, such that C is a subclass of -B, and B a subclass of A] 
and the following method m(A a, B b){a.f ; b.g; a.p();}. Let us also assume 
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that both B and C redefine method p. The corresponding CLP-decompiled code 
contains two calls to get_field/4, resp. with 'A' :f and 'B' :g. During SymEx, the 
first one will call subclass(X,'A'), which produces three alternatives (X='A', X='B' 
and X='C'). The second call to getJield will then succeed with cases X='B' and 
X='C', but fail with X='A'. Thus, the case where a and b are aliased is properly 
handled, and the calls B.pQ and C.pQ (and not A.pO) will be exercised. 

Definition 3 {symbolic execution) 

Let M be a method, m be the corresponding predicate from its associated CLP- 
decompiled program P, and P' be the union of P and the clauses in Fig. [3] with 
the described extensions. The symbolic execution of m is the derivation tree with 
root 5*0 = {m{Argsin,ArgSout,Hin,Hout,ExFlag) I 9) and = {} obtained using P' . 

The following theorem establishes the correctness of our symbolic execution mech- 
anism. Intuitively, it says that each successful derivation in the symbolic execution 
produces an output state which is correct, i.e., for any ground instantiation of such 
derivation we obtain an output state which is an instantiation of the one obtained 
in the symbolic execution. For simplicity, throughout the paper, we have included 
in an output state 6 two ingredients: the computed answer substitution a and the 
actual constraints 7. Given a constraint store 0, we say that a' is an instantia- 
tion of 6' if cr' < a and 7(7' is satisfiable. Also, we say that an output state 6' is 
an instantiation of 0, written 0' < 6, when both the corresponding stores and the 
substitutions hold the < relation. 

Theorem 1 {correctness) 

Consider a successful derivation of the form: 6*0 — > Si -^ ... -^ (e ' ^) which 
is a branch of the tree with root 6*0 = {m{ArgSin,ArgSout,Hin,Ho.ut,ExFlag) I {}) 
obtained in the symbolic execution of m. Then, for any instantiation a' of 9 which 
initializes Argsm and Hin to be fully ground, it holds that the ground execution of 
Sq ~ {m{Argsin jArgSout ,Hin jHout ,ExFlag)a' I {}) results in (e I 9') with 9' < 9. 



4 (Conditional) TCG of OO Imperative Programs 

An important problem with SymEx, regardless of whether it is performed using 
CLP or a dedicated execution engine, is that the execution tree to be traversed 
is in general infinite. In the context of TCG, it is therefore essential to establish a 
termination criterion, which guarantees that the number of paths traversed remains 
finite, while at the same time an interesting set of test-cases is generated. In addition 
to this, some approaches perform conditional TCG in which, besides selecting a 
criterion, the user establishes a precondition which further prunes the evaluation 
tree. In the remaining of this section, we describe how these issues are handled in 
our approach. 

4-1 Implementing Coverage Criteria by means of Unfolding Strategies 

A large series of coverage criteria (CCs) have been developed over the years which 
aim at guaranteeing that the program is exercised on interesting control and/or 
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data flows. Applying the coverage criteria on the CLP-decompiled program should 
achieve the desired coverage on the original bytecode. 

Implementing a CC in our approach consists in building a finite (possibly unfin- 
ished) evaluation tree by using a non-standard evaluation strategy. In (jAlbert et al. 2009| , 
we observed that this is exactly the problem that unfolding rules used in partial 
evaluators of (C)LP solve, and we proposed block-fc, a new CC for bytecode which 
was implemented with the corresponding unfolding rule. In this section, we go fur- 
ther and show that the most common CCs can be integrated in our system using 
unfolding rules. The following predicate defines a generic unfolding rule for depth- 
first evaluation strategies which is parametric w.r.t. the CC: 



unfold (Root , Goal , CCAuxDS , CCParam) : - 

select (Goal, Gieft , A, Gright ) , !, 

(internal (A) -> match(A,Bs) ; (call(A), Bs = [] ) , 

update.ccaux (CCAuxDS , A , CCAuxDS ' ) , 

append ( [Gief t , Bs , Gright ] , Goal ' ) , 

(terminates (A , CCAuxDS ' , CCParam) -> add_resultant (Root , Goal ' ) 

; unfold(Root,Goal' , CCAuxDS' .CCParam)) . 
unfold(Root,Goal,_,_) :- add_resultant (Root , Goal) . 



The main operation dependent on the CC is terminates/3, which indicates when 
the derivation must be stopped. For this aim, it uses an input set of parame- 
ters CCParam and an auxiliary data-structure CCAuxDS. Intuitively, given a goal 
Goal, an initial CCAuxDS and CCPargmis, unfold/4 performs unfolding steps un- 
til either select/4 fails, because there are no atoms to be reduced in the goal, 
or terminates/3 succeeds. In both cases, the corresponding resultant is stored, 
which can then be used to generate a test-case (or a rule in the test-case generator 
([Albert et al. 200"9| ). The Root argument carries along the root atom of SymEx. 
An unfolding step consists in the following: (1) select the atom to be reduced, which 
splits the goal into the selected atom A and the sub-goals to its left Gieft and right 
Gright! (2) match the atom with the head of a clause in the program, or call it in 
case it is a builtin or constraint; (3) update CCAuxDS; (4) compose the new goal; 
and (5) if the CC stops the derivation (i.e. terminates/3 succeeds) then store the 
resultant, otherwise (6) continue unfolding. 

In order to instantiate this generic unfolding rule with a specific CC, one has 
to provide the corresponding auxiliary data-structure and parameters, as well as 
suitable implementations for update_ccaux/3 and terminates/3. Additionally, 
match/2 and select/4 allows resp. tuning the order of generation of the evaluation 
tree, and extending the functionality of TCG by allowing non-leftmost unfolding 
steps ([Albert et al. 20(J6)) . as will be further discussed. Note that, in order to guar- 
antee that we get correct results in presence of non-leftmost unfoldings, predicates 
which are "jumped over" must be pure (see ([Albert et al. 2006]) for more details). 
E.g., for block-A:, CCParam is just the K and CCAuxDS is the ancestor stack (see 
([Albert et al. 2009| ). 
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Bench 


Es 


Cs 


Ms 


Is 


Td„ 




]^d50 


QdSO 


'-tcq 


j^d200 


Qd2DD 


'-tap 


]^bfc2 


Cbk2 


Trityp 


1 


1 


1 


98 


38 


22 


14 


100% 


20 


14 


100% 


22 


14 


100% 


Joscphus 


1 


1 


3 


61 


34 


6 


1 


56% 


366 


45 


100% 


8 


3 


100% 


Doubly LinkedList 


13 


2 


20 


253 


157 


85 


31 


37% 


594 


178 


100% 


369 


116 


100% 


RodBlackTrcc 


10 


2 


10 


485 


365 


60 


57 


30% 


2432 


539 


96% 


10010 


638 


99% 


NodoStack 


6 


3 


12 


94 


51 


14 


9 


100% 


8 


9 


100% 


8 


9 


100% 


ArrayStack 


7 


3 


11 


103 


58 


16 


15 


100% 


16 


15 


100% 


16 


15 


100% 


NodcQucuc 


6 


3 


15 


133 


73 


18 


14 


100% 


13 


15 


100% 


19 


15 


100% 


NodcDcquc 


9 


3 


19 


223 


150 


32 


23 


67% 


38 


28 


100% 


34 


28 


100% 


NodcList 


19 


9 


33 


449 


383 


152 


77 


73% 


182 


91 


91% 


184 


91 


91% 


SortodListPriorityQ 


: 11 


14 


40 


491 


442 


62 


33 


29% 


190 


79 


77% 


512 


164 


91% 


Sort 


4 


9 


30 


735 


661 


26 


12 


12% 


328 


43 


44% 


400 


55 


72% 



Table 3. Experimental results 

4-2 Including Preconditions during TOG 

In practice, it is also essential to prune horizontally the evaluation tree in order to 
limit the number of test-cases obtained without sacrificing interesting paths. The 
information used to perform this task is usually provided by the user by means of 
preconditions on the inputs, formulated using a set of pre-defined properties. These 
properties can range from simple arithmetic constraints, to more complex proper- 
ties like sharing or cyclicity of data-structures. We consider two levels of properties. 
The first-level comprises properties which can be executed beforehand thus being 
carried along by the CLP engine, like equality and disequality constraints, arith- 
metic constraints, etc. E.g., let us re-consider Ex.[3l We can specify the precondition 
that the lists are not aliased simply by providing these literals at the beginning of 
the goal "Args„ = [r(Th),L], member (L, [null,r(L')] ) , Th #\= L'". 

The second level comprises properties that require a certain level of instantiation 
on inputs in order to be executed. Depending on the property, unfold/4 can either: 
perform non-leftmost unfoldings until having the required instantiation, or incre- 
mentally check the property as the corresponding structure is being generated, or 
just delay the property check until the end of the derivation. Interestingly, the dif- 
ferent behaviors can be achieved providing suitable implementations of select/2. 
Let us re-consider again Ex. [3l We can specify the precondition that the lists do 
not share by providing this in the goal "ArgSin = [Th,L] , noshare(Th.,L)", where 
predicate noshare/2 checks that the data transitively referenced from Th do not 
share with that from L. 



5 Experimental Evaluation 

We have implemented and integrated the presented techniques in the PET tool (jAlbert et al. 20l"0|) . 

which is available for download and for online use through its web interface at 

http://costa.ls.fi.upm.es/pet. We now present some experiments which aim 

at illustrating the applicability of our approach to TCG of realistic 00 programs. 

We use two sets of benchmarks. The first group (first four benchmarks) comprises a 

set of classical programs used to evaluate testing tools taken from (jCharreteur and Gotlieb | . 
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The second one (last seven) is a selection from the net.datastructures library (|Goodrich et al. 2003[> , 

a well-known library of algorithms and data-structures for Java. Table [3] shows the 

times taken by the different phases performed by PET as well as the number of 

test-cases generated and the code coverage achieved for different CCs, block-fc and 

depth-fc (which simply limits the number of derivation steps). All times are in 

milliseconds, and were obtained as the arithmetic mean of five runs on an Intel 

Core 2 Quad Q9300 at 2.5GHz with 1.95GB of RAM, running Linux 2.6.26 (De- 

bian lenny). For each benchmark we show: the number of methods for which we 

have generated test-cases (Es); the number of reachable classes, methods and Java 

bytecode instructions (Cs, Ms and Is) (not considering Java libraries); the time 

taken by PET to decompile the bytecode to CLP (T^ec); the time of the TCG, 

total number of test-cases and code coverage for depth-50 (T^^^, N''^" and C^^"); 

for depth-200 (T^^ra^ ^^200 g^^^ Qd200^ ^^^^ ^^ block-2 (T^^^^ N^'^^ g^^^ Qbk2-^_ 

The code coverage measures, given a method, the percentage of bytecode instruc- 
tions which are exercised by the obtained test-cases, among all reachable instruc- 
tions (including all transitively called methods). This is usually the main measure 
considered in TCG to reason about the effectiveness of CCs. We observe that block- 

2 achieves a very high degree of coverage (~ 100% for the first 8 benchmarks) thus 
demonstrating its effectiveness in practice. There are however cases where block-2 
is not able to achieve 100% coverage. There are different reasons for this: (i) In 
some cases, i^ = 2 is not sufficient to reach some parts of the code. This is the case 
of most methods in class Sort. Indeed, bIock-3 achieves 100% of code coverage for 
this class, (ii) Sometimes there are parts of the code which are simply unreachable 
at execution time (dead code). This is frequent in very generic 00 programs, as it 
is the case of some methods reachable from NodeList and SortedListPriorityQ. 

The results obtained for depth-fc show that its effectiveness highly depends on 
the chosen fc, and this in turn depends on the particular program. This results in 
an unsatisfactory CC in practice. E.g., depth-50 for Joseph us obtains 1 test-case in 
6 ms, which exercises only the 56% of the code. However depth-200 achieves 100% 
coverage, but at the cost of spending much more time (366 ms), thus obtaining 
many more test-cases (45). Observe that block-2 can achieve 100% coverage with 

3 test-cases in only 8 ms. 

Overall, from the first group of benchmarks we conclude that PET can compete 
and even outperform related tools (jCharreteur and Gotlieb [ITillmann and de Halleux 2008p . 
The second group demonstrates the effectiveness of PET with realistic 00 programs 
making extensive use of inheritance and virtual invocations. A careful look at the 
most complex methods suggests that a more restrictive CC should be used to fur- 
ther prune the SymEx tree when considering more complex programs. E.g. PET 
obtains 276 (in 880 ms) for RedBlackTree.fixAfterlnsertion. We conclude also that 
the use of preconditions, as explained in Sect. 14. 2| (in principle provided by the 
user) will be crucial in order to obtain manageable test-suites for more complex 
programs. 
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6 Related Work and Conclusions 

In the fields of program verification, static analysis and static checking, transforma- 
tional approaches are widely used (Flanagan 2004 IVaziri and Jackson 2003p . The 



common technique is to translate an imperative program into an equivalent inter- 
mediate representation on which the verification, analysis or checking is performed. 



The work of ( Flanagan 2004 ) is similar to ours in the translation of the imperative 
program into a constraint logic one. However, the goal here is to perform bounded 
software model checking rather than TDG and it is not concerned with our problems 
of ensuring coverage of the shape of data structures. Also, there are no extensions 
to consider 00 features like in our work. In the case of (jVaziri and Jackson 2003() . 
the imperative program is translated into a propositional formula and SAT solving 
is used to find a solution. Again, coverage of shape of data structures is not studied 
here, which makes it fundamentally different from ours. 

Much attention has been devoted to the use of constraint solving in the au- 
tomation of software testing since the seminal work of (jDeMillo and OfFutt 1991]) . 
For the particular case of Java bytecode, (jMiiller et al. 2004]) develops a symbolic 
Java virtual machine which integrates constraint solvers and a backtracking mech- 
anism, as without knowledge about the input data, the execution engine might 
need to execute more than one path. In other approaches the problem is tackled by 
transforming the program into corresponding constraints, on which the testing pro- 
cess is then carried out by applying constraint solving techniques. Recent progress 
has been done in this direction towards handling heap-allocated data structures 
(jGotheb et al. 19981 ICharreteur et al. 20091 [Schrijvers et al. 2009p . An important 
advantage of our approach is that, since the source program is transformed into 
another (constraint logic) program -and not into constraints only- on which the 
symbolic execution is performed, we can easily track the relation between the test- 
cases and the source program. Keeping this relation is important for at least two 
reasons: (1) in order to model new coverage criteria on the source program by 
using particular evaluation techniques on the CLP counterpart, and (2) to re- 
late the generated test-cases to paths in the source program to spot errors, etc. 
This relation is less clear in pure constraint-based approaches (see discussion in 
dSchrijvers et al. 2009[ )). Some approaches are focused on improving the efficiency of 



TDG for dynamic pointer data (|Zhao and Li 200"7l Visvanathan and Gupta 2002). 



The basic idea is to separate the process of generating the shape of the data struc- 
ture to the one of generating values for the fields of data. Our approach is similar to 
them in that both process are also separated and, although actual experimentation 
is needed, we believe that a similar efficiency will be achieved. 

As another important point, while numeric data can be natively supported by 
constraint solvers, when extending the constraint-based approach to handle heap- 
allocated data structures, one has to define new constraint models based on opera- 
tors that model the heap (jCharreteur et al. 2009p . In ( [Schrijvers et al. 2009D , these 
constraints operators are implemented in CHR. In these approaches, one needs to 
adjust the solver to the particular data structures considered in the language. For 
instance, ( [Schrijvers et al. 2009[ ) provides support for lists and sketches how to ex- 
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tend it to handle trees by adding new operators. Instead, we have provided a general 
solution to generate arbitrary data structures by means of objects. 
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